Very large models
=================

RISE is built to handle **very large models** -- the parser and the post-parse
pipeline have been re-engineered so that models with several thousand
equations, shocks and parameters go through *end to end* (parse, build the
auxiliary structures, differentiate, solve). Models of upwards of ~6,000
equations, ~3,900 shocks and ~7,000 parameters parse successfully.

Why RISE requires MATLAB R2023b
-------------------------------

The scalability rests on using the right data structures. Throughout parsing,
RISE looks up variables, parameters, equations, lead/lag atoms and so on by
name an enormous number of times; doing that with linear scans
(``strcmp``/``find`` over a cell array) or with ``containers.Map`` makes the
whole pipeline grow like the *square* of the number of equations, which is
exactly the wall that used to stop large models. RISE now uses MATLAB's
``dictionary`` type for these lookups -- it gives O(1) access, supports bulk
operations, and is a value type. ``dictionary`` arrived in R2022b and
``configureDictionary`` in R2023b, which is why **R2023b is the minimum
supported release** (``rise_startup`` enforces it). On top of that, the
symbolic-differentiation backend handles sparse occurrence patterns and avoids
a "post-print" blow-up, so building the derivatives of a multi-thousand-equation
system stays tractable.

Writing a large model
---------------------

Large models are written in the usual :doc:`model language <../Model Language/rise or dsge model language>`
-- there is nothing special to declare. In practice, though, you will not type
thousands of equations by hand: use the **macro language** (the ``@#for`` /
``@#if`` loops, file includes, and the pseudo-functions) to generate the
``@endogenous`` / ``@exogenous`` / ``@parameters`` lists and the ``@model``
block compactly -- a multi-country or multi-sector model, for instance, is
naturally written as a loop over members. See the *control-flow* and
*pseudo-functions* sections of the model-language chapter.

Parsing, differentiation and memory
-----------------------------------

The first ``rise(...)`` call on a very large model is the expensive step --
parsing plus symbolic differentiation of the whole system. It is no longer
quadratic in the number of equations, but it is still substantial in absolute
terms, and it uses a fair amount of memory. A few practical points:

- Parse once, then ``save`` the model object and reload it, rather than
  re-parsing each session.
- RISE reuses computation where it can -- common-subexpression elimination in
  the symbolic differentiator, and (for block-structured models such as
  multi-country ones) it avoids re-checking subtrees across blocks that cannot
  share them.
- The differentiation backend can be chosen: ``@rsymbdiff`` (symbolic,
  arbitrary order -- the default) or ``@adolm`` (automatic, up to order 5);
  for the largest models the automatic backend can be more economical. The
  relevant solve-time options (e.g. ``solve_automatic_differentiator``) are
  documented in ``help dsge/solve``.

Solving a large model
---------------------

First-order perturbation of a model with thousands of equations is feasible --
the solvers exploit sparsity. Be aware that **higher-order** perturbation grows
quickly with the size of the state vector (the number of distinct Kronecker
columns explodes), so for the very largest models a first-order solution is the
practical choice; pick ``solve_order`` with the model size in mind. Estimation,
filtering, simulation and forecasting then proceed as usual on the solved
object.

.. todo::

   Add concrete guidance once benchmarked: typical parse / differentiation /
   solve times and memory at a few model sizes, and a worked example of a
   macro-generated large model.